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Abstract 

The  NRL  Network  Pump®,  or  Pump,  is  a  standard  for  mitigating  covert  channels  that  arise  in  a  multi-level 
secure  (MLS)  system  when  a  high  user  (HU)  sends  acknowledgements  to  a  low  user  (LU).  The  issue  here  is  that  HU 
can  encode  information  in  the  “timings”  of  the  acknowledgements.  The  Pump  aims  at  mitigating  the  covert  timing 
channel  by  introducing  buffering  between  HU  and  LU,  as  well  as  adding  noise  to  the  acknowledgment  timings. 
We  model  the  working  of  the  Pump  in  certain  situations,  as  a  communication  system  with  feedback  and  use  then 
this  perspective  to  derive  an  upper  bound  on  the  capacity  of  the  covert  channel  between  HU  and  LU  in  the  Pump. 
This  upper  bound  is  presented  in  terms  of  a  directed  information  flow  over  the  dynamics  of  the  system.  We  also 
present  an  achievable  scheme  that  can  transmit  information  over  this  channel.  When  the  support  of  the  noise  added 
by  Pump  to  acknowledgment  timings  is  finite,  the  achievable  rate  is  nonzero,  i.e,  infinite  number  of  bits  can  be 
reliably  communicated.  If  the  support  of  the  noise  is  infinite,  the  achievable  rate  is  zero  and  hence  finite  number  of 
bits  can  be  communicated. 


I.  Introduction 

A  multi-level  security  (MLS)  system  stores  and  processes  information  with  varying  sensitivity  levels  in  a  secure 
and  trusted  manner.  This  requires  that  access  to  information  is  controlled  so  that  no  high  level  information  can  be 
passed  to  users  with  lower  clearance  levels.  Thus,  a  crucial  aspect  of  design  of  MLS  systems  is  the  prevention 
of  communication  from  a  user  with  high  clearance  level,  HU,  to  a  user  with  lower  clearance  level,  LU,  such 
communication  in  an  MLS  system  is  considered  covert  [1], 
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Consider  the  communication  between  two  users,  as  shown  in  Figure  1.  The  two  users  arc  labeled  HU  and  LU, 
in  reference  to  their  respective  clearance  levels  (for  details,  see  [2]— [4]).  In  an  MLS  system,  LU  must  be  able  to 
send  packets  reliably  to  HU  and  get  a  confirmation  in  the  form  of  an  acknowledgement.  At  the  same  time,  any 
form  of  information  flow  from  HU  to  LU,  even  acknowledgements,  is  undesirable  as  it  can  potentially  be  used  to 
provide  unauthorized  data  flow  via  a  covert  channel  [1],  [5],  [6]. 

Ideally,  there  would  be  no  communication  from  HU  to  LU.  However,  acknowledgments  from  HU  to  LU  arc 
necessary  in  many  situations  for  pragmatic  reasons  of  reliability,  robustness,  and  system  performance.  Therefore, 
one  would  think  that  sanitizing  the  acknowledgements,  in  terms  of  the  data  they  carry  as  much  as  possible,  would 
suffice.  However,  the  very  timing  [7]  of  the  acknowledgements  can  open  a  communication  channel  between  HU 
and  LU.  Thus,  by  using  acknowledgements,  the  best  one  can  hope  for  is  a  quasi-secure  system,  but  it  is  still  our 
goal  to  make  the  system  as  secure  as  we  practically  can.  This  is  why  one  wishes  to  minimize  the  information  flow 
through  this  HU  to  LU  acknowledgement  covert  channel.  This  was  the  idea  behind  the  NRL  Network  Pump® 

[2] — [4]).  Here,  we  concentrate  on  the  case  of  a  single  HU  and  a  single  LU. 

Thus,  our  issue  is  that  HU  can  encode  information  in  the  “timings”  of  the  acknowledgements,  and  can  commu¬ 
nicate  with  LU  at  a  non-zero  rate.  To  avoid  this,  the  Pump  routes  the  packets  and  acknowledgements  sent  in  either 
direction  through  an  intermediate  node,  referred  to  as  the  Pump  in  Figure  1.  Even  with  this  intermediate  node,  it 
has  been  seen  [2],  [3],  and  we  will  further  see,  that  covert  timing  communication  is  still  possible,  but  at  lower 
rates.  The  Network  Pump  provides  protection  by  adding  random  noise  to  its  acknowledgement  timings  to  the  LU 

[3] . 

In  this  work,  we  consider  the  problem  from  an  information-theoretic  perspective  [8]  and  analyze  the  efficacy  of 
the  pump  in  terms  of  rate  of  information  flow  from  HU  to  LU  that  remains  available  after  deployment  of  the  pump. 
More  specifically,  we  show  that 

•  Reliable  communication  from  HU  to  LU  is  possible  even  after  deployment  of  the  pump  in  at  least  two  scenarios: 
(i)  if  the  support  of  random  noise  added  by  the  Pump  is  finite,  a  non-zero  rate  is  achievable;  (ii)  even  if  the 
support  of  random  noise  added  by  Pump  is  infinite,  still  reliable  communication  is  possible  albeit  at  zero  rate. 
In  the  case  of  noise  with  finite  support,  we  consider  the  example  of  truncated  noisy  channel  which  is  used  in 
practical  implementation  of  Network  Pump  and  characterize  the  effective  rate  of  transfer. 

•  An  upper  bound  on  the  capacity  of  the  communication  channel  from  HU  to  LU  is  derived  in  terms  directed 
information.  This  upper  bound  gives  the  worst-case  rate  of  information  leakage  between  HU  to  LU  regardless 
of  the  coding  scheme  employed. 

The  paper  is  organized  as  follows.  Related  work  is  briefly  surveyed  in  Section  II.  Notation  used  throughout 
this  paper  is  defined  in  Section  III.  In  Sections  IV  and  V,  after  mathematical  abstraction  of  the  pump,  the  covert 
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sent  by  LU  and  passes  onto  the  Pump  when  there  is  space  in  it. 

communication  over  it  is  formulated  as  an  instance  of  communication  over  a  noisy  channel  with  feedback  problem. 
In  Sections  VI  and  VII,  a  coding  scheme  for  communication  over  the  pump  is  presented  followed  by  an  upper 
bound  on  the  maximum  rate  of  information  transmission  from  HU  to  LU.  Finally,  we  conclude  in  Section  VIII. 

II.  Related  Work 

It  is  well  known  that  having  resources  shared  between  processes  can  lead  to  covert  and  side  channels  that  can  leak 
information  from  one  process  to  another.  Communication  channels  between  two  processes  that  collude  are  called 
covert  channels,  one  process  structures  its  use  of  the  shared  resource  in  a  particular  way  in  order  to  communicate 
secret  information  to  another.  Covert  channels  have  been  studied  extensively  in  the  context  of  MLS  systems,  where 
they  can  be  used  to  create  forbidden  information  flows  [9] — [13].  In  a  side  channel,  on  the  other  hand,  one  process 
tries  to  learn  something  about  the  operation  of  another  without  the  latter’s  cooperation. 

Timing  channels,  channels  where  information  is  conveyed  by  the  timings  of  messages,  arc  one  particular  class 
of  covert  and  side  channels.  They  have  been  studied  in  the  literature  in  variety  of  contexts.  In  cryptographic 
side-channels,  the  attacker  aims  at  recovering  cryptographic  keys  by  utilizing  the  timing  variations  required  for 
cryptographic  operations  [14],  [15].  The  most  common  mitigation  technique  against  such  channels  is  cryptographic 
blinding  [15],  [16].  Kopf  et  al.  derive  bounds  on  leakage  of  cryptographic  operations  using  blinding  with  quantization 
[17]. 

Transmission  of  information  by  encoding  it  in  the  timings  of  the  packets  sent  through  a  queuing  system  was 
investigated  in  [18].  In  [19],  the  authors  study  an  adversarial  queuing  setup,  where  a  jammer  has  control  of  the 
queueing  discipline.  Most  recently  Askarov  et  al.  adapting  the  periodic  quantization  methodology  of  [20]  present 
a  timing  channel  mitigation  technique  which  is  applicable  in  non  stochastic  arrival  scenarios  only.  Kadloor  et  al. 
demonstrate  a  queuing  side  channel  that  arises  in  routers  when  first-in-first-out  FIFO  or  Round  Robin  scheduling 
policies  are  used  [21].  Timing  channels  have  additionally  been  studied  in  the  context  of  language-based  security 
[22]— [24] . 

Introducing  noise  in  timing  channel  is  another  possible  mitigation  approach.  Perhaps  the  most  well  known  example 
of  this  approach  is  the  NRL  Pump  proposed  for  mitigating  timing  channels  that  arise  in  multilevel  security  systems 
(MLS)  when  a  high  confidentiality  processes  can  communicate  through  Acks  it  sends  to  a  low  confidentiality 
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processes  [25],  [26].  Past  research  on  the  Pump  [2]— [4]  gave  rough  bounds,  in  information  theoretic  terms,  for  the 
overall  covert  communication  possible  in  the  Pump.  However,  in  certain  situations,  i.e.  a  full  buffer ,  that  analysis 
did  not  take  feedback  (albeit  noisy  feedback)  into  account.  This  paper  simplifies  the  Pump  algorithm  in  order  to 
use  directed  information  to  see  how  feedback  can  influence  channel  capacity  in  the  full  buffer  scenario.  We  will 
describe  this  scenario  later  in  the  paper,  but  we  wish  for  the  reader  to  keep  in  mind  that  the  complete  Pump 
algorithm  incorporates  such  concepts  as  fair  size  to  prevent  the  buffer  from  filling.  However,  when  it  comes  to  an 
MLS  system,  all  situations  must  be  considered  before  considering  covert  communication  to  be  of  minimal  concern. 
This  paper  gives  us  such  guidance,  in  certain  situations,  and  hopefully  provides  further  insight  into  the  proper 
implementation  and  use  of  the  Pump. 

Besides  mitigation  and  quantification  efforts,  a  large  volume  of  work  on  timing  channels  has  focused  on  detecting 
timing  channels  [27],  [28], 


III.  Notations  and  definitions 


•  An  =  ( ,4 1 .  ,4 9 , . . . ,  An)  is  a  vector  of  random  variables. 

•  The  mutual  information  between  two  sequence  of  random  variables  X N  and  YN  is  defined  as  I(XN;  YN). 
It  can  be  expressed  as  (all  logarithms  are  base  2) 


I(XN-  Yn )  =  E 


'  Py„ixn(Yn\Xn) 

log  p,-»(r") 


The  directed  information,  [29],  [30],  is  defined  as 


I(X 


N 


Y jg 


N 


I>g 


Py^^mXfY^1) 


b=1  Py^AY^Y^) 

For  the  joint  random  process  {(X,,  Y)ff  \ }  the  directed  information  rate  T(X  — >  Y)  is  defined  as 


(1) 


I(X  ->  Y)  =f  limsup  —I(Xn  ->  Yr‘ 

n— kxd  Tl 


•  IT  G  W  be  a  message  in  the  set  of  equiprobable  messages  transmitted  from  HU  to  LU. 

•  The  rate  R  is  said  to  be  achievable  using  an  N-length  code  if  the  message  can  be  communicated  without  any 
error  in  N  time  units,  where: 


^  aej 


log  |W| 

N 


bits  per  transmission 


(2) 


A.  Directed  Information 

We  now  give  a  brief  explanation  of  the  role  that  directed  information  plays  in  communication  over  channels 
when  causal  feedback  is  available  at  the  encoder  (for  a  more  detailed  discussion  see  [31]).  Note  that  for  a  discrete 
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memoryless  channel  feedback  does  not  increase  capacity  [32],  However,  as  we  see  in  detail  in  Figure  4,  the  “virtual” 
encoder  we  study  is  physically  constrained  by  the  dynamics  of  the  buffer,  and  the  other  physical  interconnections. 
So  the  capacity  of  this  constrained  channel  is  unclear.  Moreover,  the  remaining  channel  has  memory,  so  feedback 
must  be  taken  into  account  for  a  capacity  analysis  when  we  arc  in  the  condition  of  a  full  buffer. 

Consider  communication  of  a  message  IF  across  a  Shannon  channel  (channel  inputs  and  outputs  are  governed  by 
the  distribution  P(Yn\Xn))  using  n  channel  uses,  without  feedback.  At  time  step  i,  the  encoder  takes  the  message 
IF  and  transmits  X,  =  ej(IF)  across  the  channel.  The  decoder  takes  the  channel  outputs  Yn  and  forms  an  estimate 
of  the  original  message  W  =  d(Yn).  To  communicate  W  reliably,  it  can  be  shown  that  the  “essence”  of  this 
problem  is  to  design  e(-)  and  subsequently  d(-)  to  maximize  the  mutual  information  /(IF;  Yn).  In  the  absence  of 
feedback,  it  can  be  shown  that  maximizing  /(IF;  Yn)  is  equivalent  to  maximizing  I(Xn:  Yn). 

If  there  is  causal  feedback  of  the  outputs  of  the  channel,  then  the  encoder  design  paradigm  is  now 

Xi  =  ei{W1Yi~l).  (3) 


With  feedback,  I(W;Yn)  can  be  re-written  as  (Figure  2): 

Pyn\W(Yn\W) 


I(W]Yn )  =  E 


log 


£e 

2=  1 
n 

£e 

2=1 

n 

£e 


2=1 


log 

log 

log 


Pyn{Y ") 

P^w^AYilW^-1) 


Py\y^(Yi\Y^) 

PYi\Yi^X\w{Yi\Yi~1,Xi,  W) 
Py^iYilYi-l) 

Py^MnY^iXi) 
Py\y^(Yi\Y^) 


=  I{Xn  ->  Yr 


(4) 

(5) 

(6) 

(7) 

(8) 


where, 

•  (5)  follows  from  the  product  rule. 

•  (6)  follows  because  X,;  is  a  function  of  IF  and  y?_1  (3). 

•  (7)  follows  because,  given  {Xl,Yl~1),  Y,  is  conditionally  independent  of  the  message  W. 

Therefore,  maximizing  /(IF;  Y"  )  is  equivalent  to  maximizing  I  (Xn  — >  F" ) .  Only  in  the  case  of  no  feedback 
arc  /(IF;  Y"  ),  I(Xn  —>  Yn),  and  / (Xn:  Yn)  equivalent.  Therefore,  directed  information  is  the  function  that  (when 
maximized)  characterizes  the  capacity  of  the  whole  channel  (the  original  noisy  channel  and  effect  of  feedback) 
between  the  original  message  IF  and  output  Yn  [33]. 
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Fig.  2.  A  communication  system  over  a  noisy  channel  with  feedback 

IV.  Working  of  the  Pump 

The  buffer  inside  the  pump  is  composed  of  two  parts.  There  is  a  buffer  b  of  size  B,  and  a  buffer  p  of  size  one. 
Each  time  LU  intends  to  send  a  packet  to  HU,  it  inserts  a  packet  into  the  buffer  p  in  the  Pump.  If  the  buffer  b  is 
not  full,  then  the  packet  is  transferred  from  buffer  p  to  buffer  b,  and  an  acknowledgement  (ACK-L)  is  sent  to  LU. 
When  the  buffer  b  is  full,  the  packet  is  retained  in  buffer  p,  and  no  acknowledgement  is  sent  to  LU.  The  packet  is 
transferred  from  p  to  b  when  some  space  clears  up. 

Whenever  the  buffer  b  is  non-empty,  the  Pump  forwards  a  packet  from  the  buffer  to  HU.  If  HU  accepts  the 
packet,  i.e.,  if  the  packet  is  transmitted  to  HU  without  any  error,  an  acknowledgement  (ACK-H)  is  sent  from  HU 
to  the  Pump.  After  receiving  ACK-H,  the  Pump  deletes  the  packet  from  the  buffer. 

It  is  important  to  note  that  the  acknowledgement  sent  by  the  Pump  to  LU,  and  the  acknowledgement  sent  by 
the  HU  to  the  Pump  serve  two  different  puiposes.  The  former  confirms  that  the  buffer  in  the  Pump  is  not  full 
and  that  the  packet  has  been  successfully  written  into  it,  while  the  latter  confirms  a  successful  read  by  the  HU.  In 
particular,  it  is  crucial  to  note  that  the  LU  does  not  know  if  and  when  the  HU  has  read  a  packet.  This  isolation 
of  the  two  users  is  the  desired  role  played  by  the  Pump. 

A.  A  covert  timing  channel  through  the  Pump 

As  stated  in  the  description  above,  the  actions  of  HU  and  the  actions  of  LU  are  isolated  by  the  Pump.  However, 
there  is  one  scenario  in  which  the  Pump  fails  to  provide  this  isolation.  This  is  the  case  when  the  buffer  in  the  Pump 
is  full.  Note  two  things:  Lirst,  the  full  buffer  channel  was  studied  in  [3],  but  that  was  done  without  feedback  being 
taken  into  consideration — so  those  results  may  be  interpreted  as  being  overly  optimistic;  hence  the  need  for  this 
paper.  Secondly,  the  operation  of  the  Pump  in  its  usual  mode  of  not  having  a  full  buffer  still  may  leak  information 
from  HU  to  LU.  This  covert  leakage  has  been  estimated  in  prior  Pump  work,  and  is  not  the  subject  of  this  paper. 
Here  we  are  only  concerned  with  the  covert  channel  that  arises  due  to  a  full  buffer. 

Consider  the  case  when  LU  sends  a  packet  to  the  Pump  when  the  buffer  is  full.  In  this  scenario,  the  packet  is 
written  into  buffer  p.  The  packet  will  get  transferred  from  the  buffer  p  to  buffer  b  only  when  the  latter  has  space. 
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Fig.  3.  A  systems  theory  diagram  denoting  the  joint  interaction  between  all  the  observed  and  unobserved  state  variables.  The  arrows  in  black 
show  the  direction  in  which  packets  flow  through  this  network.  LU  and  HU  are  the  low  clearance  and  the  high  clearance  user  respectively. 
p  is  one  additional  unit  of  space  where  the  last  transmitted  packet  from  LU  is  cached  before  passing  it  onto  the  Buffer  inside  the  pump. 
The  dashed  blue  arrow  is  drawn  to  indicate  how  the  state  of  the  buffer,  whether  it  is  full  or  not,  influences  the  working  of  p  and  the  noise 
added  at  the  pump. 


Hence,  unlike  when  the  buffer  b  is  not  full,  LU  does  not  get  an  acknowledgement  immediately.  It  will  receive  an 
acknowledgement  as  soon  as  HU  successfully  reads  a  packet  from  the  Pump,  thereby  deleting  a  packet  from  the 
buffer.  Thus,  LU  knows  exactly  the  timings  when  HU  reads  the  packets  and  therefore,  in  this  scenario,  HU  can 
communicate  to  LU  by  encoding  a  message  in  the  time  it  sends  acknowledgements  to  the  buffer. 

To  avoid  the  above  mentioned  scenario,  the  Pump  adds  a  random  time,  based  on  a  moving  average  of  HU 
acknowledgement  times,  to  every  acknowledgement  that  it  sends  to  LU.  We  analyze  a  simplified  version  (the 
moving  average  is  held  constant)  of  this  system  to  study  if  the  addition  of  this  ‘noise’  to  the  acknowledgements 
guarantees  an  acceptably  minimal  covert  communication  rate  from  HU  to  LU. 


B.  Dynamics  of  the  process 

The  complete  dynamics  of  the  system  are  given  below  and  presented  in  Figure  3.  The  thick  black  lines  denote 
the  direction  of  physical  transmission  of  the  packets  and  the  acknowledgements.  The  dotted  blue  lines  represent  the 
virtual  feedback  present  because  of  the  dynamics  of  the  system.  This  becomes  clear  from  the  following  description 
of  the  system. 

System  model:  We  denote  by  br  and  />, ,  the  state  of  the  respective  buffers  b  and  p  at  time  i.  The  state  of  a 
buffer  is  the  number  of  packets  in  the  buffer.  We  assume  that  the  time  is  discretized,  so  i  =  0, 1, 2,  3, ... .  We  will 
further  assume  that  all  the  links  are  reliable,  i.e.  there  is  no  packet  loss  in  transition. 

Transmission  of  packets  from  the  LU:  Let  i  be  the  binary  valued  random  variable  indicating  whether  a 
packet  was  transmitted  at  time  i.  Note  that  we  use  the  notation  xt-  \  and  not  xt  to  denote  the  packet  sent  at  time  i 
as  we  will  later  interpret  {xi}  as  being  the  feedback  from  LU  to  HU.  LU  sends  a  packet  to  the  Pump  and  waits 
for  ACK-L.  It  sends  the  next  packet  only  after  receiving  ACK-L  for  the  previous  packet. 
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Insertion  of  packets  into  the  Pump  :  If  the  LU  sends  a  packet  at  time  i,  it  is  first  written  into  the  buffer  p. 
If  the  buffer  b  is  not  full,  then  the  packet  is  immediately  written  onto  the  buffer  b  and  the  packet  is  cleared  from 
buffer  p.  However,  if  the  buffer  b  is  full,  then  the  packet  is  retained  in  buffer  p. 

Acknowledgements  sent  to  LU:  If  a  new  packet  is  sent  by  LU  at  time  i,  and  if  it  is  successfully  written  into 
the  buffer  6,  then  the  Pump  prepares  to  send  ACK-L  to  LU.  Note  that  a  successful  insertion  of  the  packet  into  the 
buffer  b  happens  only  when  the  buffer  is  not  full. 

Random  back-off  for  the  transmission  of  acknowledgements:  Once  the  Pump  is  ready  to  send  ACK-L,  it  waits 
for  a  random  number  of  time  slots  before  it  sends  ACK-L  out.  Let  us  denote  by  qi,  the  number  of  packets  at 
time  i  which  are  not  acknowledged  by  the  Pump.  We  can  think  of  such  packets  waiting  in  a  queue,  whose  service 
times  arc  distributed  according  to  some  distribution.  Then  qi  is  the  ‘state’  of  that  queue  at  time  i.  Once  the  random 
back-off  time  expires,  an  acknowledgement  is  sent  to  LU  immediately.  Let  z%  denote  the  departure  process  from 
this  queue. 

Constraint  on  the  transmissions  of  packets  by  LU:  LU  is  not  allowed  to  transmit  the  next  packet  to  the  Pump 
until  it  has  received  ACK-L  for  the  packet  transmitted  previously.  This  constraint  implies  that  qt,  which  is  the 
number  of  packets  which  arc  not  yet  acknowledged  by  the  Pump,  can  at  most  be  one. 

Transmission  of  packets  from  the  Pump  to  HU:  Whenever  the  buffer  b  is  non-empty,  the  Pump  forwards  a 
packet  to  HU.  This  is  denoted  by  the  process  {gi},gi  G  {0, 1}.  The  packet  is  not  erased  from  the  buffer  though. 
The  buffer  waits  for  an  acknowledgement  from  HU  before  transmitting  the  next  packet. 

Acknowledgement  from  HU:  After  the  packet  is  successfully  received  by  HU,  it  sends  ACK-H  to  the  Pump, 
given  by  {hi},  hi  G  {0, 1}.  After  receiving  ACK-H,  the  Pump  erases  the  packet  from  the  buffer.  It  is  important  to 
note  that  HU  controls  the  time  when  ACK-H  is  sent  to  the  Pump. 

V.  Communication  over  a  channel  with  feedback 

Consider  Figure  4  which  captures  the  system  model  developed  so  far.  LU  sends  a  stream  of  packets  {j;,;},  where 
xi  G  {0, 1}  is  the  indicator  if  a  packet  has  been  sent  in  time  slot  i. 

The  buffer  p  can  be  thought  of  as  a  queue,  whose  states  are  zero  and  one.  The  input  to  this  queue  happens 
through  the  arrival  process  {a;,  },  and  a  departure  happens  whenever  the  buffer  is  not  full.  Denote  { y, }  as  the  output 
process  of  the  buffer  p.  Thus, 


Vi 


Vi 


fy(Pi—l  7  Xi—  i,  bf) 

1  pi-i  =  D,Xi-i 
<  1  Pi- i  =  l,Xj_l 


1 ,1(bi  =  B)  =  0. 

(9a) 

0 ,1{bi  =  B)  =  0. 

(9b) 

0  otherwise. 
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Virtual  Encoder  Channel  Decoder 


Fig.  4.  System  drawn  as  communication  system  over  a  channel  with  feedback  as  described  in  Sec  IV-B. 


Fig.  5.  A  canonical  communication  system  with  feedback.  The  LU  (Figure  4)  acts  as  a  decoder,  the  Pump  plays  the  role  of  a  noisy  channel, 
and  the  dynamics  of  the  system  along  with  the  actions  of  the  HU  act  as  a  virtual  encoder. 

The  number  of  packets  in  the  buffer  is  given  by  6*.  The  indicator  function  I (br  =  B)  is  one  if  the  buffer  b  is 
full,  zero  otherwise.  The  buffer  p  is  a  unit  memory  buffer.  Let  p,  denote  the  state  of  the  buffer  at  time  i.  Dynamics 
at  buffer  p  is 


Pi  =  fp(pi-i,bi,Xi-i) 


(10a) 


Po 


Pi 


0 

f  1  pi-i  =  0,Xi-i  =  l,I(bi  =  B)  =  1. 
0  otherwise. 


(10b) 

(10c) 


Note  that  when  the  buffer  is  not  full,  yt  =  x',_ i .  If  yt  /  0,  then  a  new  packet  gets  written  into  the  buffer,  and  b, 
increments  by  one.  The  process  {y*}  serves  as  an  input  to  the  queue  of  the  Pump.  Inputs  to  buffer  b  are  through 
the  departure  process  from  buffer  p,  and  departure  process  through  the  sending  of  acknowledgements  from  HU. 
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=  h-  hi  +  yi . 
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(11a) 

(lib) 


The  Pump  takes  the  process  {yi},  and  after  a  random  service  time,  lets  out  the  packets,  denoted  by  the  process 
{zi}.  In  the  real  system,  this  mimics  the  process  of  the  Pump  taking  in  data  packets  from  LU  and  transmitting  the 
acknowledgement  packet  after  a  random  delay.  Thus,  the  Pump  acts  as  a  FCFS  queue  with  a  noisy  service  time. 

qi  represents  the  number  of  packets  at  time  i  which  have  not  been  acknowledged.  The  input  to  this  queue  is 
through  the  departure  process  from  buffer  p.  The  departure  from  this  queue  depends  on  whether  the  random  back-off 
time  has  expired  or  not.  The  queue  dynamics  can  hence  be  written  as 


Qi+i  =  fq  {qi,zi,yi+ 1) 

(12a) 

=  qi  -  Zi  +  yi+ 1, 

(12b) 

where  Zi  is  the  random  variable  that  denotes  whether  an  acknowledgement  has  been  sent  or  not. 

We  assume  that  the  decoder  (LU)  inserts  a  new  packet  into  the  system  as  soon  as  it  receives  acknowledgements 
for  all  the  packets  it  has  transmitted  so  far. 


Xi  =  fx{xl  X,zl) 

(13a) 

(  1  Efc=1*fc-1  =  Ylk=lZk- 

(13b) 

Xi  =  < 

I  0  otherwise. 

Each  time  the  buffer  is  not  empty,  it  sends  a  packet  to  HU.  This  is  captured  in  the  process  {p*}  £  {0, 1}.  The 
actions  of  HU  is  a  function  of  the  packets  he  receives,  {<■/,  },  and  the  message  W,  which  the  HU  wishes  to  convey 
to  LU.  It  is  assumed  that  the  buffer  sends  a  new  packet  to  HU  as  soon  as  it  has  received  an  acknowledgement  for 
the  previous  packet  (and  if  the  buffer  is  non-empty).  The  output  process  of  the  queue  at  HU  affects  number  of 
packets  in  the  buffer.  This  output  process  models  the  acknowledgement  packets  sent  by  HU,  which  result  in 
packets  being  erased  from  the  buffer. 


II 

5k 

o- 

(14a) 

/ 1 

Efc= i  9k  =  Efc=i  hki  hi  >  0. 

(14b) 

9i  =  \ 

l  0 

otherwise. 

hi  =  fh{gl • 

,  W) 

(14c) 

We  will  state  a  brief  lemma  which  we  will  use  later. 
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Lemma  5.1:  The  variables  yn  and  bn  are  deterministic  functions  of  zn.  given  the  message  W  and  initial  state  of 
the  buffer  b.  Bo. 

Proof:  Follows  from  equations  (9),(10),(1 1),  and  (14).  ■ 

A.  Deviations  from  the  original  Pump 

What  we  have  presented  above  is  a  simplification  of  the  actual  Pump  algorithm  in  the  following  ways.  This  has 
been  done  to  examine  the  entire  suite  of  possible  covert  communication  from  HU  to  LU,  and  to  make  sure  the 
special  cases,  such  as  the  full  buffer,  arc  adequately  addressed. 

The  actual  Pump  sends  messages  to  LU  based  upon  an  off-set  and  truncated  exponential  random  variable  with 
mean  based  upon  a  moving  average  of  past  HU  acknowledgement  times.  In  this  paper,  we  assume  that  the  noise 
added  by  the  pump  is  a  random  variable  whose  statistics  do  not  change  with  time.  This  approximation  is  necessary 
in  order  to  quantify  fundamental  limits  on  the  capacity  of  the  covert  channel.  Moreover,  in  many  communication 
systems,  i.i.d.  noise  is  the  worst  possible  form  of  noise.  We  conjecture  that  it  is  the  same  for  Pump  as  well,  although 
we  do  not  conclusively  justify  it. 

Another  deviation  from  the  original  pump  setup  is  that  this  paper  does  not  include  the  concept  of  ’’Fair  size” 
(sec.  3.1.3,  [2]).  The  concept  of  Fair  Size  is  a  design  parameter  intended  to  keep  the  queue  length  at  a  desired 
level.  This  desired  level  will  ensure  that  even  if  the  flow  of  input  packets  is  bursty,  there  is  enough  space  for  all 
the  packets.  The  choice  of  the  Fair  size  in  [2]  is  subjective.  This  concept  is  not  relevant  in  our  problem  formulation 
because  in  our  setup,  the  traffic  from  the  Low  User  is  not  bursty.  Infact,  the  Low  User  is  always  active  and  assumed 
to  have  the  next  packet  ready  for  transmission  as  soon  as  it  receives  the  acknowledgement  for  the  previous  packet. 
Finally,  we  did  not  consider  packet  drops  in  our  modeling. 

B.  Differences  between  other  communication  channels  with  feedback 

Network  pump  is  similar  to  trapdoor  problem  [34]  and  exponential  server  timing  channel  [35]  to  the  extent  of 
the  following:  both  the  trapdoor  channel  and  exponential  server  timing  channel  can  be  represented  as  channels 
with  linear  dynamics  and  internal  feedback,  [36],  [37].  We  are  motivated  from  those  examples  to  use  directed 
information  upper  bounds  for  the  amount  of  information  flow  through  the  system.  Having  said  that,  the  Network 
pump  setup  is  significantly  different  from  the  above  two  examples  in  the  following  aspects: 

In  the  exponential  server  timing  channel,  the  queue  state  dynamics  are  linear.  In  the  case  of  Network  Pump,  the 
dynamics  are  very  complex.  Compare  Figure  12  of  [36]  to  Figure  2.  The  input  to  the  channel  X,  in  the  case  of 
exponential  server  timing  channel  is  a  linear  function  of  the  message  W  and  feedback  Yl~l .  In  the  case  of  Network 
Pump,  the  same  relationship  is  a  composition  of  the  functions  given  in  the  equations  (8),(9),(10),  and  (14)  -  which 
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is  clearly  non-linear  and  more  complex.  Please  note  that  there  arc  differences  in  the  notations  used  in  [36]  and  this 
paper. 

The  actions  of  the  High  User  and  the  Low  User  who  play  the  role  of  the  encoder  and  the  decoder  respectively 
are  restricted.  The  High  User  can  only  acknowledge  packets  which  have  already  been  sent  by  Low  User  and  the 
Low  User  can  only  send  packets  after  it  has  received  an  acknowledgement  from  the  pump.  This  is  very  different 
from  the  traditional  communication  systems  in  which,  other  than  an  average  power/rate  constraint,  the  possible 
actions  of  the  encoder  and  decoder  are  in  no  way  restricted  by  the  channel. 

Because  of  these  differences,  the  subsequent  analysis  is  fundamentally  different. 

VI.  A  SIMPLE  TRANSMISSION  STRATEGY 

We  present  a  coding  scheme  which  illustrates  that  a  covert  communication  channel  can  indeed  be  created  from 
HU  to  LU.  We  start  the  analysis  at  time  t  =  0  when  the  buffer  is  assumed  to  be  empty: 

Phase  1:  LU  sends  B  packets  addressed  to  HU  back  to  back.  That  is,  the  first  packet  is  sent  from  LU  at  t  =  0 
and  an  ack  is  received  at  t  =  N\,  where  N,t  is  the  noise  added  by  the  pump  for  the  acknowledgement  to  the  ?th 
packet.  The  second  packet  is  sent  at  t  =  N\  +  1  and  an  ack  received  at  t  =  iVi  +  N2,  and  so  on.  The  pump,  after 
receiving  the  packet  forwards  it  to  the  HU.  However,  in  phase  1,  HU  does  not  ack  any  packet  that  he  receives. 
Packets  would  therefore  get  accumulated  inside  the  buffer  of  the  pump.  This  stage  lasts  for  t  =  BK  time  slots, 
where  K  is  a  coding  parameter  chosen  later.  The  LU  and  HU  assume  that  the  buffer  is  full  at  this  time.  This  will 
be  true  if  LU  has,  by  this  time,  received  ACKs  from  the  pump  for  all  his  B  packets.  We  will  shortly  show  that  this 
assumption  is  true  when  K  is  chosen  to  be  a  suitably  large  value.  After  this  time,  if  LU  tries  to  send  any  more 
packets,  they  will  get  dropped. 

Phase  2:  At  this  time,  HU  can  start  communicating  to  LU  by  selecting  the  time  when  he  reads  packets  from 
the  buffer.  LU  sends  one  packet  every  time  slot  starting  t  =  BK.  They  are  dropped  at  the  pump  because  the  buffer 
inside  the  pump  is  full.  However,  as  soon  as  HU  acks  one  of  the  B  packets  which  arc  already  in  the  buffer,  the 
pump  accepts  one  new  packet  from  LU. 

Starting  at  time  t  =  BK,  HU  waits  for  a  random  time  M \  before  sending  an  ack  for  one  of  the  packets  already 
in  the  buffer.  Hence,  at  time  t  =  BK  +  Mi,  one  space  in  the  buffer  clears  up.  Recall  that  LU,  in  Phase  2,  is 
continuously  trying  to  send  a  packet.  The  packet  sent  by  LU  at  time  t  =  BK  +  Mi  +  1  is  therefore  written  into 
the  buffer.  The  pump  sends  an  ack  to  LU  at  time  t  =  BI\  +  AI\  +  1  +  N^+\. 

Mi  is  a  discrete  valued  random  variable.  For  the  time  being,  we  will  assume  that  Mi  has  finite  support  set. 
HU  can  choose  a  distribution  for  Mi  that  will  maximize  the  rate  of  information  transfer.  We  will  comment  on  this 
distribution  later  on. 
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This  phase  lasts  for  duration  K  as  well.  At  time  t  =  (B  +  1  )K,  HU  assumes  that  LU  has  received  an  ack  for 
the  B  +  1th  packet.  He  then  waits  for  a  random  time  M2  and  then  reads  a  packet  from  the  buffer,  and  the  process 
continues  as  detailed  in  Phase  2. 

In  Phase  2,  LU  transmits  packets  at  each  time  slot,  HU  reads  packet  i  at  time  t  =  (B  +  i  —  1)K  +  Mt,  and  LU 
gets  an  ack  for  the  packet  B  +  i  at  time  t  =  (B  +  i  —  1)K  +  M,-  +  1  +  A-  Note  that  in  making  this  statement,  we 
have  inherently  assumed  that  Af  +  A  <  K,  we  need  to  prove  this. 

1)  Analysis  of  probability  of  decoding  error:  Consider  a  time  horizon  of  t  G  [BK  +  1,  (/i  +  n)K]  time  units 
divided  into  n  blocks  of  K  time  units  each.  Let  {M,  G  Al}f=1  take  values  in  {1,  2,  •  •  •  ,  A! j }  where  the  support 
size  Ni 1  is  finite.  We  consider  the  analysis  of  error  probability  for  two  different  cases 

•  the  noise  A  is  geometric  with  mean  E[A]  = 

PN(Ni  =  k)  =  (1  -p){k-1]p,  k  =  1,2,3,- ••  (15) 

•  the  noise  Ay  is  truncated  geometric  of  rate  //  and  truncated  at  K'  where  the  support  K'  and  duration  of  each 
block  in  the  strategy  K  satisfy  K'  +  \M\  =  K. 

* e {1,2 

(16) 

0  otherwise. 

If  Mi  +  Ni  <  K,\/i  <G  {1,2, .. .  ,n},  then  the  channel  essentially  behaves  like  an  additive  noise  channel.  In  this 
scenario,  the  probability  of  decoding  error  can  be  shown  to  go  to  0  by  invoking  Asymptotic  Equipartition  Property 
(AEP)  [32], 

Since  the  truncated  geometric  case  is  a  special  case  of  geometric  noise,  we  analyze  the  geometric  noise  case 
first.  Further  analysis  corresponds  to  geometric  noise  unless  otherwise  specified. 

For  the  dynamics  of  the  system  given  above,  define 

•  The  transmission  times  of  the  HU  as  {A*  =  (B  +  i  —  1  )K  +  Mj } ,  and  the  receiver  times  of  the  LU  as 
A:  =  {{B  +  i  -  1  )K  +  Mi  +  1  +  Ni}. 

.  K'  =  K-\M\, 

•  Define  event  E$  to  occur  if  the  time  taken  in  the  first  phase  to  fill  the  Pump  to  its  full  capacity  B  is  less  than 
BK  time  units. 

•  Define  event  Ay  to  occur  when  A  <  (B  +  i  +  1  )K.  Define  Ef  =  (n"=1  Ay)  H  Eq.  Event  E\l  is  equivalent 
to  saying  for  each  i.  Mi  +  Ni  <  K. 

•  Define  event  Ay  to  occur  if  A  <  K  —  \A4  =  K'  and  A  =  (H”=1Ay)  H  A-  Since  the  support  size  of  Mi 
is  bounded  above  by  \M\,  Ni  <  K' .  V  i  implies  Mi  +  A  <  K.  V  i.  Thus  event  Ay  implies  event  Ay  and 


Copyright  (c)  2011  IEEE.  Personal  use  is  permitted.  For  any  other  purposes,  permission  must  be  obtained  from  the  IEEE  by  emailing  pubs-permissions@ieee.org. 


This  article  has  been  accepted  for  publication  in  a  future  issue  of  this  journal,  but  has  not  been  fully  edited.  Content  may  change  prior  to  final  publication. 


14 


Eg  C  E\ .  Moreover,  if  event  E%  occurs,  then 

(B  +  i  —  1  )(K)  <  Ai  <  Di  <  (B  +  i)K,\/i  =  1,2,  •  •  •  ,n  (17) 


and  hence  the  communication  in  a  given  block  is  independent  of  dynamics  of  the  past,  conditioned  upon 
event  E%.  Hence,  the  proposed  achievable  scheme  converts  communication  over  this  channel  with  dynamics 
to  n-channel  uses  over  an  additive  noise  channel  with  input  Ml  and  output  Mi  +  ATt  conditioned  upon  event 
Enx. 

Lemma  6.1:  The  probability  of  event  E™  occurring  is  given  by 


'(E?)  =  (i-(i-riK'ni-(i-riK») 

log  (n) 


1,  as  n  — >  oo  if  K'  > 


log(l  -  n) | 

Moreover,  the  conditional  joint  distribution  of  noise  N.t  conditioned  upon  event  EJJ'  is  given  by 


(18) 

(19) 


IP  (All  =  a1;  Al2  =  a2,  ■ 

••  ,Nn  =  an\E2) 

=  < 

rnua-Mp*  v 

di  <  K',\/  i  =  1,2,-- 

•  ,n 

(20) 

Proof:  Refer  Appendix-A. 

o, 

V 

otherwise. 

■ 

Lemma  6.2:  For  a  block  length  n,  such  that  each  Mt  ~  Pm,  Ay  ~  given  in  (15),  and  decoder  has  access  to 
{Mi  +  Ni}2=i ,  R  given  by  R  <  H(M  +  N)  —  H(N ),  there  exist  a  block  code  such  that 


|W| 

> 

en(R—ei(n)) 

(21a) 

p  (w  ±  w) 

< 

e2(n) 

(21b) 

PCil  =  P  (w  /  W\E%) 

< 

(n) 

(21c) 

where  ei(n),  e2(n),  £3(71)  — >  0  as  n  — >  00  if  n  and  K’  satisfy  K'  >  |  log^f— |  - 

Proof:  Refer  Appendix-B.  ■ 

Corollary  6.3:  For  a  block  length  n,  such  that  each  Mi  ~  Pm,  Ay  ~  P/v  given  in  (16),  and  decoder  has  access 

to  {Mi  +  Ay}”=1,  R  given  by  R  <  H(M  +  N)  —  H(N),  there  exist  a  block  code  such  that 

|W|  >  en^R~e^n))  (22a) 

P  (w^W^J  <  e2(n)  (22b) 

where  ei(n),e2(n)  0  as  n  — >  00. 


Copyright  (c)  2011  IEEE.  Personal  use  is  permitted.  For  any  other  purposes,  permission  must  be  obtained  from  the  IEEE  by  emailing  pubs-permissions@ieee.org. 


This  article  has  been  accepted  for  publication  in  a  future  issue  of  this  journal,  but  has  not  been  fully  edited.  Content  may  change  prior  to  final  publication. 


15 


Proof:  For  the  truncated  noise  given  in  (16),  the  event  Ef  occurs  with  probability  1,  IP  {Ef)  =  1.  Hence, 


P  (w  f  W\E ^  =  P  (w  +  W^j  <  e2(n) 


(23) 


With  our  problem  setup,  it  is  not  always  possible  for  the  LU  to  have  access  to  {Mi  +  JVj}"=1.  To  avoid  this  we 
separate  communication  instants  long  enough  so  that  LU  has  access  to  {Ml  +  iVj}”=1. 

Theorem  6.4:  For  the  problem  setup  of  Network  pump,  such  that 

•  each  Mi  ~  Pm,  Ni  ~  Pjy  given  in  (15),  R  given  by  R  <  H(M  +  N)  —  H(N), 

•  h  =  nK  time-instants  are  used  in  total,  where  n  and  K  satisfy 


K 


log(rc) 

log(l  -  fi)\ 


+  \M\ 


there  exist  a  code  such  that  \W\  number  of  messages  can  be  distinguished  where 


(24) 


\W\  >  en(*-ei(n))  (25a) 

p(w  f  lu)  <  e4(n)  (25b) 

where  e\ (?r),  e4(n)  ->  0  as  n  ->  oo. 

Proof:  Refer  Appendix-C.  ■ 

Corollary  6.5:  For  the  problem  setup  of  Network  pump,  such  that 

•  each  Mi  ~  Pm,  Ni  ~  Pn  given  in  (16),  R  given  by  R  <  H(M  +  N)  —  H(N), 

•  n  =  nK  time-instants  are  used  in  total,  where  the  support  of  noise  K'  and  duration  of  each  block  in  the 
strategy  K  satisfy  K'  +  |PW  =  K  as  defined  in  (16), 

there  exist  a  code  such  that  |W|  number  of  messages  can  be  distinguished  where 


|W|  >  en(R_ei(n))  (26a) 

P  (w^W^J  <  e2(n)  (26b) 

where  e4(?r),  e4(?r)  — >  0  as  n  — >  oo. 

Proof:  Since,  Mi  +  Ni  <  K  for  all  i,  event  E->  holds  true  with  probability  1.  Hence,  from  corollary  6.3 

P  (w  ±  =  P(w  fW\El^)=  e2(n)  (27) 
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Thus,  from  Thm  6.4,  we  can  be  able  to  distinguish  between  \W\  number  of  messages  given  by  (25a)  with  error 
tending  to  0.  The  effective  rate  of  transfer  will  be 

~  nR  R 

R=  —  <  .  ,  t - >0 

nK  l°sH _ ,\M\ 

Hence,  the  effective  rate  of  transfer  is  0  while  promising  reliability.  Though  the  support  of  noise  is  infinity,  we 
were  able  to  find  reliable  codes  such  that  a  finite  number  of  messages  can  be  transmitted. 

For  the  truncated  geometric  noise  case,  where  the  noise  support  is  finite,  the  effective  rate  is  given  by 


^  nR  R 

Rtrunc  =  ^K  =  K'  +  \M\ 


>  0 


Hence,  if  the  noise  added  by  the  pump  Nt  has  finite  support,  then  we  can  transmit  infinite  number  of  bits  or 
equivalently  a  non-zero  rate  while  promising  reliability.  Note  that  the  effective  rate  Rtrunc  depends  on  the  value 
of  R  =  H(M  +  N)  —  H(N).  While  the  entropy  of  noise  N  for  the  truncated  geometric  noise  with  parameters  /< 
and  K'  is  fixed,  the  entropy  of  M  +  N  can  be  maximized  to  improve  the  rate  of  transfer.  We  will  now  analyze  the 
characteristics  of  this  rate  Rtrunc  as  a  function  of  supports  |At|  and  K' . 


A.  Simulation  results  for  Truncated  Noise 

The  mutual  information  maximizing  input  distribution  for  an  additive  noise  channel  with  truncated  geometric 
noise  is  not  easy  to  compute  in  closed  form.  We  hence  resort  to  simulations.  We  use  the  Blahut-Arimoto  algorithm 
(refer  Chapter  10  of  [32]  for  details)  to  compute  it. 

The  rate  at  which  information  is  conveyed  from  HU  to  LU  is  then 

—I(M:  N)  =  - - - I(M;  N) 

I<  v  ’  1  \M\  +  K'  v  ’  ; 

In  Figure  6,  we  plot  this  rate  as  a  function  of  the  support  size  At.  The  rate  achieved  is  of  the  form  ). 

For  a  given  value  of  the  support  set  of  noise  N,  there  is  a  best  value  of  At  which  results  in  highest  rate.  If  this 
value  of  At  is  used  by  the  encoder,  the  rates  obtained  arc  plotted  in  Figure  7.  We  also  plot  the  tradeoff  between 

forward  rate  (from  LU  to  HU)  and  the  covert  communication  rate  (from  HU  to  LU)  in  Figure  8.  For  this  plot,  K' 

was  fixed  at  100,  the  mean  of  the  truncated  geometric  variable  was  varied  and  the  covert  communication  rate  was 
computed.  The  forward  rate  is  the  inverse  of  the  mean  of  the  truncated  geometric  noise.  The  main  focus  of  the 
paper  is  to  demonstrate  that  covert  communication  over  a  network  Pump  is  possible,  and  hence  we  do  not  provide 
a  detailed  analysis  of  this  tradeoff. 
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u  =  0.9 


Fig.  6.  Plot  of  the  variation  of  rate  as  a  function  of  the  support  set  for  Mi,  which  is  \M\.  For  a  fixed  parameter  /r.  the  effective  rate  of 
transfer  varies  in  the  order  O(lo^4^f'  ). 


Fig.  7.  Maximum  rate  at  which  HU  can  communicate  to  LU  as  a  function  of  the  support  set  of  the  noise,  K' .  This  plot  characterizes  the 
‘effective  rate  of  transfer’  ( Rtmnc )  vs  the  support  of  noise  added  by  Pump  to  the  acknowledgements  sent  to  LU  (K1). 
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Fig.  8.  The  Forward  rate  is  the  rate  at  which  the  packets  are  transmitted  from  LU  to  HU.  The  covert  communication  rate  is  the  rate  from 
HU  to  LU.  The  support  set  of  the  noise  was  fixed  at  100  for  this  simulation. 


VII.  Upper  bound:  HU  to  LU  transmission  rate 

Let  R  be  the  rate  of  transmission  (2)  from  HU  to  LU.  The  receiver  (LU),  determines  and  observes  the  sequences 

(n) 

Xn  and  Zn  respectively,  and  uses  only  these  sequences  to  estimate  W.  Let  P,  be  the  probability  of  making  an 
error  P  (w  /  W^j ,  where,  Wn  =  f{Xn.  Z")  is  the  estimate  of  W  at  the  receiver.  We  say  that  rate  R  is  achievable 

if  P  (w  /  Wn  -»■  o). 

Theorem  7.1:  If  R  is  achievable,  then  II  <  T(y,  B  —>  Z\Bq) 

Proof:  Since  R  is  achievable,  Fano’s  inequality  implies 

H(W\Xn,  Zn)<  1  +  P^n)nR.  (28) 
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Hence,  if  Pe ^  —>  0  as  n  — ■>  oo,  then  ^H(W\Xn,  Z" )  — >  0.  Thus,  if  the  error  in  decoding  is  to  go  to  zero,  then 
the  uncertainty  in  W  at  the  receiver  should  certainly  go  to  zero. 


=  H(W)  =  H{W\B0) 

(29) 

=  H(W\Xn,  Zn ,  B0)  +  I(W\  Xn,  Zn\Bo) 

=  H(W\Xn,  Zn,  B0)  +  I(W ;  Zn\B0) 

+  I{W-Xn\Zn,Bo) 

=  H(W\Xn,  Zn ,  B0)  +  I(W ;  Zn\B0) 

(30) 

<  H{W \Xn,  Zn)  +  I(W-  Zn\B0) 

(31) 

<l  +  P^nR  +  I{W-Zn\Bo) 

(32) 

where  (29)  follows  because  the  message  to  be  transmitted  is  assumed  independent  of  the  initial  state  of  the  buffer, 
(30)  follows  because  Xn  is  a  deterministic  function  of  Zn,  (31)  follows  because  conditioning  reduces  entropy,  and 
finally,  (32)  follows  from  (28).  Using  Bayes’  rule 


I{W]Zn\B0)  =  E 


log 


P 


W\Z™,B  o 


P 


W\B0 


=E 


log 


p 


Z^\W,B0 


P. 


Z™\B0 


(33) 


n 

Pzn\W,B0  =  \\_PZi\Z^,W,B0 
i— 1 


(34a) 


n 


i=l 

n 

(34b) 

1~J  PzfZ'-i^Y^Bi 

i=l 

(34c) 

where  (34a)  follows  by  chain  rule,  (34b)  is  because  of  Lemma  5.1,  (34c)  follows  because  our  channel  is  non- 
anticipative  (refer  Figure  5),  or  equivalently,  the  Markov  relation  W  —  (if ,  bfz1^1)  —  Zi  holds.  With  this,  we  have: 


I(W;Zn\B0)= 


i= 1 


log 


PZi\z*-\w  {ZilZ'-fW) 


EE 

i=  1 


log 


Pz^z1-1  ,b0  {Zi\Zl  1,B0) 

PZi\Z‘-\YKB‘  {Zj\Z^\Y\B*) 
PzAZ i~1,B0  {ZilZ1-1,  Bq) 


=  ^  I(Y\  Bl- ZfZ*-1,  B0)  =  I(Yn,  Bn  ->  Zn\B0) 


(35a) 
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where  (35a)  follows  from  (33)  and  (34);  Thus  the  information  rate  from  the  HU  to  LU  is  represented  in  terms  of 
directed  information  flow  over  the  system  dynamics.  From  (32),  (35) 


nR  <1  +  P^nR  +  I(W;  Zn\B0 ) 

=1  +  P^nR  +  I(Yn,  Bn  ->  Zn\B0 


(36a) 

(36b) 


Dividing  by  n  and  let  n  — »  oo,  we  conclude 


R  <  - I(Yn ,  Bn  -»•  Zn\B0)  =  I(y,  B  -»•  Z\Bq) 
n 


A.  Significance  of  Directed  Information  Upper  Bound: 

We  have  so  far  shown  that  the  directed  information  rate  from  (Y.  b)  to  Z  is  an  upper-bound  on  the  rate  of 
communication  over  this  channel.  The  tightness  of  this  bound  is  yet  to  be  investigated  and  can  only  be  verified  by 
demonstrating  an  achievable  scheme.  The  directed  information  expression  over  the  channel  can  then  be  formulated 
as  an  equivalent  dynamic  programming  problem  and  approximate  solution  can  be  found  just  like  in  the  case  of 
ii ni filar  Finite  State  Channel  (FSC)  [34]  and  the  feedback  capacity  problem  [38].  The  directed  information  bound  is 
proven  to  be  tight  for  similar  examples  of  Trapdoor  channel  with  feedback  [34]  and  an  Exponential  Server  Timing 
Channel  (ESTC)  [35].  We  will  discuss  the  example  of  ESTC  where  the  directed  information  bound  is  tight. 

Consider  the  communication  system  in  Figure  9.  An  ESTC  can  be  interpreted  as  a  special  case  of  such  a  system 
[36].  In  an  ESTC,  where  time  is  slotted  finely  enough  to  ensure  that  in  a  given  time  slot  of  duration  A,  there  can 
be  at  most  one  arrival.  Xj,  is  the  number  of  packets  in  the  queue  at  time  k,  Z  is  the  arrival  process  (chosen  by 
the  encoder),  Y  is  the  departure  process,  and  //(•)  is  the  update  law  of  the  queue,  A/,.  =  Xy._\  +  Z}.  —  Y).__  \ .  The 
memoryless  channel  is  a  Z-channel,  P(Yj.  =  1|  A/.  f  0)  =  7A  and  P(Yj.  =  1|  A/,.  =  0)  =  0,  where  7  is  the  mean 
service  time  of  the  ESTC.  Similar  to  our  problem,  directed  information  is  an  upper  bound  to  the  communication 
rate  in  an  ESTC,  and  also,  a  rate  equal  to  directed  information  can  be  achieved  over  the  channel  when  using  Poisson 
inputs  [35].  It  is  in  essence  because  the  dynamical  system  that  is  unknown  to  the  encoder  but  does  operate  on 
previous  outputs  of  the  noisy  channel,  is  in  some  sense  optimally  using  that  feedback,  along  with  the  encoder’s 
input  process.  In  the  case  of  communication  over  the  Pump,  the  encoder  (HU)  does  have  some  information  about 
the  state  (the  encoder  knows  if  the  buffer  B  is  empty  or  not),  and  hence,  is  in  some  sense  even  perhaps  better  off 
than  the  exponential  server  timing  channel.  We  might  therefore  hope  that  the  directed  information  bound  reasonably 
approximates  the  actual  rate,  if  not  tightly,  and  our  future  efforts  will  be  directed  in  developing  achievable  schemes. 
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Fig.  9.  Communication  over  a  channel  with  dynamics 

VIII.  Conclusions,  Discussion  and  Future  Work 
We  have  analyzed  the  capacity  of  the  covert  channel  present  when  a  Pump  is  used  to  isolate  communication  between 
two  users  (HU  and  LU)  with  different  clearance  levels,  and  the  Pump  buffer  becomes  full.  Following  a  careful 
modeling  of  its  working,  we  show  that  it  is  possible  to  communicate  over  this  channel.  The  HU-LU  communication 
is  interpreted  as  communication  over  a  channel  with  noisy  feedback,  and  we  provide  theoretical  bounds  on  the  rate 
of  communication.  The  upper  bound  is  nonconstructive  and  is  in  terms  of  a  directed  information  over  the  parameters 
of  the  system.  However  because  of  similarity  of  the  problem  at  hand  to  the  Trapdoor  channel  with  feedback  [34] 
and  the  Exponential  Server  Timing  Channel  (ESTC)  [35],  we  will  not  surprised  if  in  fact  the  directed  information 
bound  can  be  proven  to  be  tight.  In  fact,  proving  the  tightness  of  the  upper  bound  is  an  interesting  direction  for 
future  work. 

Our  lower  bounds  on  the  capacity  of  the  channel  between  HU  and  LU  are  constructive  and  we  present  an 
achievability  scheme  which  guarantees  non-zero  communication  rate  (infinite  bits)  if  the  noise  added  by  the  pump 
has  a  finite  support.  Even  if  the  pump  adds  random  noise  with  infinite  support  to  the  ACKs  from  Pump  to  LU 
still  reliable  communication  is  possible  albeit  at  zero  rate,  i.e,  finite  number  of  bits  can  be  transmitted.  As  depicted 
in  Figure  7,  the  achievable  communication  rate  is  a  function  of  support  of  noise  added  by  pump.  It  is  noteworthy 
that  by  increasing  the  support  of  noise,  not  only  Pump  can  reduce  the  effective  rate  of  communication  between 
HU  and  LU,  but  also  it  will  affect  the  the  QoS  of  the  system  as  it  will  slow  down  the  legitimate  communication 
rate  between  LU  and  HU.  In  short,  depending  on  the  QoS  and  security  requirements  of  a  system  (the  tolerable 
communication  rate  from  HU  to  LU),  Pump  must  vary  its  noise  support. 
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Appendix  A 
Proof  of  Lemma  6. 1 

The  probability  of  event  E\'  occurring  is  given  by 


'(E;')  =  p(n?,,Eun£o) 
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where  (37)  follows  from  the  chain  rule  of  probability,  (39)  holds  true  because  by  conditioning  that  all  the  receiver 
times  I)}.  <  (B  +  k)(K  +  1)  till  (i-1)  transmissions,  and  thus  does  not  interfere  with  the  communication  in  the  i-th 
block.  In  other  words,  79, -l  <  {B  +  i)(K)  <  A{  <  Di,  and  the  receiver  times  {79/c}(~11  does  not  interfere  with 
the  transmission  in  the  i-th  block.  Thus,  the  communication  for  the  i-th  transmission  is  independent  of  the  past. 
Hence  E\j  conditioned  on  k  =  1, 2,  •  •  •  ,  (i  —  1)  having  occurred  is  equivalent  to  M,;  +  Nr  <  K.  (39)  follows 
because  Ay  <  K'  =>•  M,  +  Nt  <  K,  and  so  P ( Mj  +  Nj  <  I\)  >  P (Ay  <  K').  (40)  follows  from  the  properties 
of  geometric  noise  (15).  (42)  holds  if  K'  >  |  i0gfi-|t)| *  i-e-’  grows  at  a  rate  greater  than  |  iQg(i— | ' 

Similarly,  we  can  compute  the  conditional  joint  distribution  of  noise  N,  conditioned  upon  event  E%  using  the 
conditional  independence  of  communication  between  blocks  when  Ay  <  K' . 

P  (A^i  =  ai,  N2  =  a2,  ■  ■  ■  ,Nn  =  an\E%) 


P  (Ay  =  01,  N2  =  02,  •  •  •  ,  Nn  =  an\E%  n  E%) 
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0,  otherwise. 


where  (43)  follows  because  E%  C  E\l  by  definition  of  E%  and  E™.  (44)  follows  from  chain  law  of  probability, 
(45)  follows  from  the  independence  between  blocks  conditioned  upon  E2,  (46)  follows  from  conditional  law  of 
probability,  (47)  follows  from  the  properties  of  geometric  distribution.  Hence,  the  noise  is  equivalent  to  a  truncated 
geometric  noise  of  rate  p  and  truncated  at  K'  when  conditioned  upon  event  E%. 
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Appendix  B 
Proof  of  Lemma  6.2 

The  proof  of  (21a)  and  (21b)  follows  from  standard  coding  theorem  [32,  Chapter  8].  For  (21c), 
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where  (49a)  follows  from  conditional  law  of  probability,  (49b)  is  because  probability  of  an  intersection  of  events 
is  less  than  a  single  event,  (49c)  follows  from  (21b),  (49d)  follows  from  properties  of  geometric  distribution  and 
(49e)  holds  when  K'  >  logi  /n_m(n). 


Appendix  C 
Proof  of  Theorem  6.4 

(25a)  follows  from  (21a)  of  Lemma  6.2.  (25b)  can  be  proved  as  follows: 
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where  (50a)  follows  from  the  additive  law  of  probability,  (50b)  follows  because  probability  of  any  event  is  less 
than  1,  (50c)  follows  from  (21c)  in  Lemma  6.2,  (50d)  follows  from  the  properties  of  the  geometric  distribution 
where  K'  =  K  —  \M\,  (50e)  holds  true  if  K  >  |  iogfi-^t)|  +  |-M|- 
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